Demystifying Legacy Migration Options to the AWS Cloud

By Phil de Valence, Solutions Architect for Legacy Modernization at AWS

Many companies or institutions still possess legacy non-x86 systems in their datacenters: mainframe, midrange, or UNIX proprietary systems.

Migrating these workloads across hardware architectures to the Amazon Web Services (AWS) Cloud requires advanced software technology. Short-term migration approaches mainly use hardware emulation, middleware emulation, automated refactoring, or middleware replatforming.

This post highlights short-term migration options, their key technical differences, as well as their differentiated benefits. These options are particularly suited for custom home-grown applications running on a legacy platform where the complete source code is available. On the other hand, third-party packaged applications generally require discussing with the third-party software vendor for modernization options to AWS.

Short-Term Migration Options

These migration options result in shorter project duration for faster return on investment (ROI) and quick wins.

The diagram in Figure 1 shows, for each option, components that change and components that stay the same during the migration of the non-x86 legacy platform.

Figure 1 – Legacy migration component changes.

• Legacy Hardware Emulation: The hardware emulator replaces the legacy hardware but the legacy operating system (OS) and applications stay the same.
  .
• Legacy Middleware Emulation: The middleware emulator replaces legacy middleware APIs and OS APIs required by the application, allowing for porting. Majority of the application source code is recompiled without changes, with some adaptations for changed dependencies.
  .
• Legacy Automated Refactoring: Code, data, and dependencies are automatically converted to a modern language, data store, and frameworks, while guaranteeing functional equivalence with the same business functions.
  .
• Modern Middleware Replatforming: This applies only to modern languages, middleware, and runtimes that are available across legacy and x86 systems such as Java, PHP, and relational databases. It allows for reuse of the application code and databases.

Differentiated Value Propositions

Each migration option has its preferred use-cases and unique value proposition, relevant for a specific legacy workload.

Considering an average-sized workload with a few million lines of code, the diagram in Figure 2 shows key differentiating characteristics, like project cost, project duration, and cloud agility, as represented by the Cloud-Native Maturity Model. This model promotes use of AWS Managed Services, Twelve-Factor App design principles, Microservices, and automation such as Continuous Integration/Continuous Deployment (CI/CD).

Figure 2 – Migration options relative to duration, cost, and agility.

Details and explanations about each migration option are provided in the subsequent sections.

Legacy Hardware Emulation

• Migration type: Re-host using hardware emulation.
  .
• Applicable systems: This option applies to IBM Z mainframes, Unisys ClearPath mainframes, SPARC machines, AlphaServer computers, VAX stations, HP 3000 and HP 9000 computers, PDP minicomputers, and others. It allows running operating systems such as IBM z/OS, Unisys ClearPath MCP or OS2200, Solaris, SunOS, HP-UX, MPE, OpenVMS, Tru64, RSX-11, RSTS, or RT-11, on top of AWS infrastructure.
  .
• Description: The vendor emulator software emulates legacy hardware instruction set. Therefore, it can run the legacy binaries on top of a modern Linux and x86-64 instruction set. It allows running the legacy OS and its applications unchanged with no need to recompile source code or to transform binaries.

Figure 3 – Legacy hardware emulation..

• Target stack example: Vendor hardware emulators run on Amazon Elastic Compute Cloud (Amazon EC2) instances. Because of the legacy OS, AWS Managed Services are not used for compute or data.
  .
• Migration approach: Because the source OS, binaries, and files are unchanged, this is a lift-and-shift re-hosting of the legacy OS from the legacy hardware to the software emulator. It’s typically done with a backup-restore or dump-restore of the legacy OS, and transferring the backup files over AWS Direct Connect or AWS Snowball. The applications, data, and interfaces are unchanged, and the amount of required testing is reduced, allowing migrations in days or weeks.
  .
• Typical use cases: Offloading development and test environments to AWS; Migrating stabilized production applications to AWS; Keeping the same application development software stack while modernizing the hardware; Datacenter shut down including legacy hardware decommissioning.
  .
• Hardware emulator benefits: No change to application code, middleware and operating system; Same interfaces and same communication protocols; Migration transparent to end-users and application owners.
  .
• AWS benefits: Large choice of Amazon EC2 instance types for CPU, memory, storage, network bandwidth; Capacity changes and scalability in minutes; Optimized cost with AWS pay-as-you-go model or Reserved Instances (RI); Higher availability across Availability Zones (AZ).
  .
• Caution: Emulating hardware instruction sets requires on-the-fly instruction translation, which has a performance impact. Depending on the source legacy hardware age and depending on the emulator software, this performance impact can be more or less noticeable. Either way, a performance benchmark is recommended. Amazon EC2 instances with the fastest processor clock speed, such as the z1d instances at 4.0 GHz, can help.
  .
• Vendor examples: Stromasys Charon, IBM Z Development and Test Environment, Unisys ClearPath Forward.
  .
• Vendor best practices: It’s important to understand how each vendor optimizes the emulator instructions translation and possible performance tuning options with AWS.

Legacy Middleware Emulation

• Migration type: Re-platform with code ported to a middleware emulator.
  .
• Applicable systems: This option applies to some IBM Z mainframes, Unisys ClearPath mainframes, IBM AS/400, iSeries, and IBM i. It allows porting and recompiling applications previously relying on legacy middleware and data stores, such as IBM CICS, IMS, VSAM, QSAM, Unisys COMS, TIP, IBM DB2 for z/OS, DB2 for i (DB2/400), and Adabas. These applications can be written in languages such as COBOL, PL/I, RPG, and Natural.
  .
• Description: The vendor emulator software emulates middleware and operating system APIs necessary for the application code. For example, middleware emulators can provide the APIs for indexed files access, for transaction management, for legacy screens and protocols support, for temporary storage, and batch support. This allows porting and recompiling the source application code for native x86-64 execution.
  .
  If a specific language is not supported, it needs to be converted to a supported one first. Because of the recompilation, there’s no on-the-fly instruction translation performed by the emulator, and consequently no related performance impact. Dependencies or utilities not provided by the emulator (scheduler, print, security, etc) need to be replaced with x86 equivalents with likely corresponding code or configuration adaptations.
  .
  The legacy data format is either retained (VSAM, QSAM) or mapped to a relational data store such as Amazon Aurora, Amazon RDS for Oracle, or Amazon RDS for SQL Server.

Figure 4 – Legacy middleware emulation.

• Target stack example: Vendor middleware emulators typically run on Amazon EC2 instances. Some dependencies for security, scheduling, printing, or message queuing are deployed in their own instances. The data stores can be modernized to benefit from Amazon RDS managed services including Amazon Aurora. Some applications can benefit from AWS Auto Scaling and Elastic Load Balancing across AZs.
  .
• Migration approach: Because of the OS, middleware and dependency changes, code adaptations, data format conversion, and complete recompilation, this is a replatforming project where the programs and business functions are incrementally migrated and thoroughly tested.
  .
  It starts with the discovery of all programs, data, dependencies, and integrations. It continues with designing the target architecture and new dependencies, defining the programming language conversions and replacements, and planning the work packages and the transitions. Then, the bulk of the migration happens with automated mass changes to source code, updates to the code migration rules, corresponding data conversion, and testing.
  .
  Tests focus on changes of code, data, dependencies, and integrations. Once unit, regression, integration, and performance tests are successful, application and data are deployed in their target AWS environments. More details are available in Migrating a Mainframe to AWS in 5 Steps and in a customer example in Complete Mainframe to AWS Migration with Candid Partners. Such migration can last between months and years depending on the number of lines of code and other factors.
  .
• Typical use cases: Migrating away from proprietary and expensive legacy hardware and software stack; Retaining the application code investment, keeping the development team, while modernizing the underlying infrastructure; Offloading development and test environments to AWS; Migrating stabilized production applications to AWS.
  .
• Middleware emulator benefits: Eliminate legacy vendor software and hardware; Preserve the application programming language, code, and development skills; Same interfaces and same communication protocols; Migration transparent to end-users; Facilitate the modernization and integration with cloud services; Reduce risks by preserving the same logic and code.
  .
• AWS benefits: Includes the AWS benefits described for the hardware emulator, plus the ability to use AWS Managed Services such as Amazon RDS or Elastic Load Balancing. Some applications can benefit from horizontal scalability and elasticity with Auto Scaling Groups. Applications can benefit from DevOps services to manage a CI/CD pipeline.
  .
• Caution: Such projects are not lift-and-shift. It requires detailed analysis and planning for dependency changes, code adaptations, data conversion, and thorough testing. Some legacy application code or legacy data formats can make it challenging to use AWS best practices for scalability and availability.
  .
  If a language or asset is not supported by the emulator (assembler, generated code, proprietary databases), the complexity increases drastically. Applications rely mostly on the middleware emulator for AWS services integration. Different middleware emulator vendors have different capabilities for AWS integrations and optimizations.
  .
• Vendor examples: Micro Focus Enterprise Server, NTT DATA UniKix, TmaxSoft OpenFrame, Astadia OpenMCS, Infinite i.
  .
• Vendor best practices: Applications benefit from better availability, elasticity, performance, and cost optimization when using AWS horizontal scalability. Middleware emulators should facilitate horizontal scalability, especially for legacy data stores, shared in-memory data structures, favoring cross-AZ active/active architectures and avoiding singletons.
  .
  Middleware emulators also differentiate themselves with support for AWS features such as Amazon Aurora, Amazon RDS, Amazon Linux, and Amazon CloudWatch.

Legacy Automated Refactoring

• Migration type: Automated refactoring with conversion of code, dependencies, and data.
  .
• Applicable systems: This option applies to languages such as COBOL, PL/I, RPG, Natural, and FORTRAN, including code generators such as Cool:Gen or PowerBuilder applications. It converts data models from data stores such as IBM DB2 z/OS, VSAM, QSAM, IMS DB, DB2 for i (DB2/400), Adabas, and Data Management System.
  .
  That means many applications hosted on IBM Z mainframes, Unisys ClearPath mainframes, IBM AS/400, iSeries, and IBM i can benefit from automated refactoring to AWS.
  .
• Description: Automated refactoring is not a manual re-write of code, but an automated conversion and generation of modern code, data access, data format, and dependencies altogether. This exhaustive and coherent transformation guarantees functional equivalence while automation increases speed and reduces risks.
  .
  This transformation applies patterns and rules for transforming legacy components, such as legacy screens, indexed files access, and batch to a modern target stack with object-oriented and service-oriented applications, choice of language (Java or C#), choice of backend frameworks (e.g. Spring), and frontend frameworks (e.g. Angular).
  .
  Automated refactoring tools give you, right out of the box, the ability to choose various underlying AWS services while transforming to cloud-native applications. The refactoring can also isolate groups of programs and dependent data, facilitating the identification and creation of microservices.

Figure 5 – Legacy automated refactoring.

• Target stack example: By default, vendor tools convert the stack to run on Amazon EC2 instances for compute, on Aurora or Amazon RDS for datastore, forging a typical elastic web architecture. With the flexibility of automated refactoring and the ability to tune the transformation rules, many other AWS Managed Services can be incorporated, such as Amazon Simple Queue Service (SQS) for messaging or Amazon ElastiCache for in-memory shared data structures.
  .
  Some applications can also benefit from executing in containers within Amazon Elastic Container Service (Amazon ECS) and Amazon Elastic Container Service for Kubernetes (Amazon EKS), or within serverless AWS Lambda functions. Because the new stack is service-enabled, business functions are easily published to new clients such as mobile applications via Amazon API Gateway.
  .
• Migration approach: The overall approach and phases of a legacy automated refactoring project are similar to the ones of a legacy middleware emulation project, because they both include an inventory of assets, mapping the application dependencies to the new stack components, mass changes to the source code, code compilation, and extensive testing, which can take 60-80 percent of the time spent on the project.
  .
  More specifically, automated refactoring tools allow reverse engineering of the application code and data models in order to build an application model showing all program and data dependencies, along with their key characteristics. This detailed model is used to define the migration strategy, components and data mappings, code and data format conversion rules, work packages decomposition, and transitions.
  .
  The bulk of the migration happens with automated conversions of code and data, with incremental transitions, updates to the conversion rules, extensive testing, and without manual code rewrite. More details are available with customer examples in Automated Refactoring of a New York Times Mainframe to AWS with Modern Systems, and in Automated Refactoring of a U.S. Department of Defense Mainframe to AWS. Such overall migration can last months or years depending on the number of lines of code and other factors.
  .
• Typical use cases: Migrating away from languages such as COBOL, PL/I, RPG, and Natural; Modernizing and standardizing the complete technical stack while keeping the application business functions investment; Solving the retirement and legacy skills gaps; Innovating with agile practices, managed services, and cloud-native technologies.
  .
• Automated refactoring benefits: Eliminate legacy technologies; Preserve the application business functions and similar interfaces; Reduce or eliminate middleware software licensing costs; Application and infrastructure agility; Transform to a stack similar to a cloud-native application; Facilitate microservices definition and peeling; Reduce risks by guaranteeing functional equivalence.
  .
• AWS benefits: The modern application architecture allows better horizontal scalability, elasticity, cost efficiency, and DevOps best practices. Data becomes easily reusable across the many AWS big data and analytics services. The service-enabled programs can be quickly integrated with new use-cases such as mobile application via Amazon API Gateway or voice interfaces with Amazon Alexa.
  .
• Caution: The automated refactoring tool is a critical success factor. Such tools must be evaluated thoroughly during a complex Proof-of-Concept (PoC) demonstrating functional equivalence, conversion speed, tests speed, code quality, and code performance. Batch can be a good test-case for the complex PoC.
  .
  Automated refactoring projects involve extensive testing. This can impact the project price and duration drastically depending on which automation tools are being used, what the tests scope is, how thorough the tests are, and who is responsible for executing these tests.
  .
• Vendor examples: Blu Age, TSRI, Modern Systems, Heirloom.
  .
• Vendor best practices: Many vendors provide reverse-engineering tools for inventory and deep-dive analysis of legacy assets. But few vendors provide forward engineering with high automation (no manual code re-write), comprehensive program and data coherent conversion (not just line-by-line transcoding), large AWS services support (right tool for the right job), and quality maintainable code.
  .
  Maintainable code should eliminate legacy COBOL idioms like GO-TOs and PERFORMs and MOVEs (No JOBOL). It should also reduce code complexity (which can be measured by SonarQube), use native language operators and types and methods, allow multi-threading for performance, and adapt to customer naming conventions and coding standards.

Modern Middleware Replatforming

• Migration type: Replatforming, keeping the same modern middleware but different operating systems.
  .
• Applicable systems: This option applies to middleware or runtimes that are available similarly (typically with the same code base) for both the legacy operating system and the x86 operating system. For example, it applies to applications relying on IBM WebSphere Application Server, on PHP or Perl runtimes, on Java virtual machines, or on IBM DB2 databases.
  .
• Description: Because the same middleware is available across operating systems, the application code and data can be moved and executed with minimal or no changes. For example, interpreted languages are platform independent. On the database side, the same data schema is reused or adaptations are made to migrate to a different database type.

Figure 6 – Modern middleware replatforming.

• Target stack example: Such replatforming can use, depending on the middleware software support and prerequisites, Amazon EC2 instances and AWS Managed Services such as Amazon Elastic Beanstalk, Amazon EKS, Aurora, and Amazon RDS.
  .
• Migration approach: Because the applications and data rely on specific middleware from third-party vendors, specific migration procedures and instructions from the same vendors must be followed in order to stay in a supported migration path. Such procedure can involve export then import of application packages, or backup then restore of databases including schema and data.
  .
• Typical use cases: Standardize the software technical stack; Move to managed services; Leave expensive proprietary operating systems.
  .
• Modern middleware Replatforming benefits: Eliminate legacy operating systems; Enable infrastructure agility with elasticity, instance types, DevOps best practices, and infrastructure-as-code.
  .
• Caution: There can be challenges if the application uses native OS functions or if the middleware does not support the same feature set across platforms.

Other Options You May Consider

Manual rewrite requires redeveloping business logic from scratch and manually in new programming language, and re-architecting the infrastructure services. For typical legacy applications with millions of lines of code, such an approach has higher risks:

• Trying to capture the old logic specifications.
• Bugs with manual code development.
• Business functions inconsistencies.
• Testing functions which are not like-to-like anymore.
• managing very large political projects with many people over a 4-5 years period of time.

Manual rewrite is better suited for basic functions, and for pinpoint transactions or programs with clear limited scope, interfaces, and data.

Repurchase with a software package involves identifying and replacing the legacy home-grown application with a third-party packaged application. This is typically better suited for applications whose core logic is better developed by a third-party vendor. For example, a SaaS payment solution or SaaS industry-specific supply chain solution.

Depending on whether the legacy application executes core-business functions or not, standardizing to an undifferentiated software package could result in losing competitive advantage. With such projects, there are risks around trying to capture business rules from the legacy application, and there are risks around the escalation of manual developments and manual customizations required to meet the business needs.

Migration vs. Augmentation

Migration is about incrementally but permanently moving applications from a legacy platform to AWS, and eventually shutting down the legacy hardware. However, some customers want to keep the legacy platform for now, and still benefit from AWS value proposition. These customers can augment their legacy platform with AWS services.

For example, we have customers successfully using AWS for legacy data analytics, and for enabling new interfaces such as mobile or voice. These customers typically use data replication between the legacy data store and AWS data services.

Other customers choose to offload and scale development and test environments on AWS. These typically use an Integrated Development Environment (IDE) on AWS, use emulators described previously for tests, and promote code with a pipeline back to the legacy platform for production.

And then there are other customers that replace their expensive backup and archival storage (such as virtual tape hardware) with cost-optimized Cloud storage backup solutions.

Quality of Service

Legacy systems often have stringent non-functional requirements. AWS offers numerous services and features to build and operate secure, high-performing, resilient, and efficient systems supporting enterprise workloads. For example: security data encryption with AWS Key Management Service (KMS); reliability with AZs and Regions; elasticity with Auto Scaling Groups; system management with AWS Systems Manager and Amazon CloudWatch.

The AWS Well-Architected Tool and AWS Trusted Advisor help build and operate secure, efficient, and cost-effective systems in the cloud. Furthermore, AWS adds agility to workloads by providing cloud speed, managed services, large choice of services, and facilitated innovation.

Migration Approach

There is no one-size-fits-all for legacy modernization. Customers need the right tool for the right job, aligned with both the IT strategy and legacy technical constraints. Because of the technical complexity, any approach definition must include a hands-on technical Proof of Concept confirming if the approach and tool recommendations are viable.

When considering an assessment for a legacy workload migration, the following deliverables facilitate the decision process:

• Inventory of assets to be migrated.
• Complex PoC proving technical viability.
• Detailed description of the migration approach.
• Phased project plan for complete migration.
• AWS sizing and project cost estimates.
• Business case for legacy migration including ROI.

To learn more about legacy to AWS migration capabilities, please reach out to your AWS representative and review our partner solutions on the APN Blog.